{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mnist Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset contain gray-scale images of hand-drawn digits, from zero through nine.\n",
    "\n",
    "Each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Importing Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading Mnist dataset from the Internet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.datasets import mnist\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train features: (60000, 28, 28)\n",
      "Train Labels:   (60000,)\n",
      "Test Features:  (10000, 28, 28)\n",
      "Test Labels:    (10000,)\n",
      "Dimention:  3\n",
      "Dimention:  1\n",
      "Dimention:  3\n",
      "Dimention:  1\n"
     ]
    }
   ],
   "source": [
    "print(\"Train features:\",x_train.shape)\n",
    "print(\"Train Labels:  \",y_train.shape)\n",
    "print(\"Test Features: \",x_test.shape)\n",
    "print(\"Test Labels:   \",y_test.shape)\n",
    "print(\"Dimention: \",x_train.ndim)\n",
    "print(\"Dimention: \",y_train.ndim)\n",
    "print(\"Dimention: \",x_test.ndim)\n",
    "print(\"Dimention: \",y_test.ndim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualizing Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAOFElEQVR4nO3dbazU5ZnH8d9vXY1P9QE5IrEqLdFE0ghtJsboxripNGpMsMZqiUE3mtIXPrSmJouuWqMvwI1tU4Opnq5auhRIpRJPoq4FYjR90zCaszwIK6hoqSgHMWhfYFe99sUZNkc8c89hnvH6fpKTmflfc8//yoQf/5m5Z/63I0IAvvz+odcNAOgOwg4kQdiBJAg7kARhB5L4x27ubPLkyTFt2rRu7hJIZfv27dq9e7fHq7UUdtuXSPqlpMMk/UdELCrdf9q0aapWq63sEkBBpVKpW2v6ZbztwyQ9LOlSSTMkzbU9o9nHA9BZrbxnP1fStoh4IyL+LmmFpDntaQtAu7US9lMl/WXM7R21bZ9je77tqu3qyMhIC7sD0IpWwj7ehwBf+O5tRAxGRCUiKgMDAy3sDkArWgn7Dkmnjbn9VUnvtNYOgE5pJezrJJ1p+2u2j5D0fUlD7WkLQLs1PfUWEZ/YvlnS8xqdens8Ija1rTMAbdXSPHtEPCvp2Tb1AqCD+LoskARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IImWlmy2vV3SR5I+lfRJRFTa0RSA9msp7DX/HBG72/A4ADqIl/FAEq2GPST90fbLtuePdwfb821XbVdHRkZa3B2AZrUa9gsi4luSLpV0k+0LD7xDRAxGRCUiKgMDAy3uDkCzWgp7RLxTu9wlaZWkc9vRFID2azrsto+x/ZX91yV9R9LGdjUGoL1a+TR+iqRVtvc/zrKI+K+2dIWD8uGHH9atLViwoDh206ZNxfqaNWuK9cMPP7xYR/9oOuwR8YakmW3sBUAHMfUGJEHYgSQIO5AEYQeSIOxAEu34IQw6bOnSpcX6XXfdVbf29ttvt7Tv0rSeJJ100kktPT66hyM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBPHsf2LFjR7F+2223Feu7d9c/32ftJ8hNu+WWW4r1xYsXF+uTJk1qaf9oH47sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE8+x94MEHHyzW33///S518kUrVqwo1p977rlivfRb+0Zz+EcccUSxjoPDkR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCevQveeuutYv2JJ55o6fFnzqy/mO6UKVOKY1evXt3Svvfu3Vusl75DcO211xbHnnLKKU31hPE1PLLbftz2Ltsbx2ybZHu17a21yxM72yaAVk3kZfxvJF1ywLYFktZGxJmS1tZuA+hjDcMeES9J2nPA5jmSltSuL5F0RZv7AtBmzX5ANyUidkpS7fLkene0Pd921XZ1ZGSkyd0BaFXHP42PiMGIqEREZWBgoNO7A1BHs2F/z/ZUSapd7mpfSwA6odmwD0m6vnb9eklPt6cdAJ3ScJ7d9nJJF0mabHuHpJ9KWiTp97ZvlPS2pO91sslD3fDwcLHeaA30Cy+8sFh/8cUX69b27dtXHLts2bJifeHChcX6tm3bivV33323bm3OnDnFsY1+K8856Q9Ow7BHxNw6pW+3uRcAHcTXZYEkCDuQBGEHkiDsQBKEHUiCn7h2wccff1ysN1pWudGSzSVHHnlksX7DDTcU6ytXrizWX3/99WI9IurWjj766OJYTiXdXhzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJ5tm7YPny5S2Nf+aZZ4r1K67o3CkAq9Vqxx77vPPOK9aPPfbYju07I47sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE8+xdMHduvRP0jnr66fJp99etW1esb9mypW5tw4YNxbGrVq0q1j/44INi/YQTTmh6/ODgYHHsvHnzivUZM2YU6/g8juxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATz7F1w8cUXF+vHH398sb5+/fpi/eyzz65ba3RO+kZmz55drD/88MPF+uWXX1639tprrxXHPvTQQ8X6I488Uqzj8xoe2W0/bnuX7Y1jtt1r+6+2h2t/l3W2TQCtmsjL+N9IumSc7b+IiFm1v2fb2xaAdmsY9oh4SdKeLvQCoINa+YDuZtvray/zT6x3J9vzbVdtV0dGRlrYHYBWNBv2X0maLmmWpJ2SflbvjhExGBGViKgMDAw0uTsArWoq7BHxXkR8GhGfSfq1pHPb2xaAdmsq7Lanjrn5XUkb690XQH9oOM9ue7mkiyRNtr1D0k8lXWR7lqSQtF3SDzvY4yFv0qRJxfqTTz5ZrF911VXF+t69e+vWSuujS9Ktt95arD/wwAPFeqP136+88sq6tYULFxbHPv/888V6o7Xhp0+fXqxn0zDsETHemRce60AvADqIr8sCSRB2IAnCDiRB2IEkCDuQBD9x7QONfgK7cuXKYn3ZsmV1a41O9XzfffcV642m1hq5++6769Y2b95cHNvoFNuNel+yZEmxng1HdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1Ignn2Q0CjefhG9V466qij6tauueaa4thG8+wvvPBCsb5nT/1TJzb62fGXEUd2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCeXb0zNVXX12sDw0NFesrVqwo1hcvXly3ds899xTHfhlxZAeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJNxoSd92qlQqUa1Wu7Y/HNqGh4eL9fPPP79Y37dvX93ali1bimPPOuusYr1fVSoVVatVj1dreGS3fZrtF2xvtr3J9o9q2yfZXm17a+3yxHY3DqB9JvIy/hNJP4mIsyWdJ+km2zMkLZC0NiLOlLS2dhtAn2oY9ojYGRGv1K5/JGmzpFMlzZG0f32dJZKu6FSTAFp3UB/Q2Z4m6ZuS/ixpSkTslEb/Q5B0cp0x821XbVdHRkZa6xZA0yYcdtvHSvqDpB9HxIcTHRcRgxFRiYjKwMBAMz0CaIMJhd324RoN+u8i4qna5vdsT63Vp0ra1ZkWAbRDw5+42rakxyRtjoifjykNSbpe0qLaZfm8v8BBmjVrVrF+//33F+u333573dodd9xRHLt06dJivXSK7H41kd+zXyBpnqQNtvdPfN6p0ZD/3vaNkt6W9L3OtAigHRqGPSL+JGncSXpJ325vOwA6ha/LAkkQdiAJwg4kQdiBJAg7kASnksYh67rrrivWH3300bq1p556qm5NkrZu3Vqsn3POOcV6P+LIDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM+OQ1ajMx+tWbOmbu2MM84ojl20aFGxvmzZsmK9H3FkB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkmGfHl9bpp59etzZ79uzi2KGhoWL91VdfLdZnzJhRrPcCR3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSGIi67OfJum3kk6R9JmkwYj4pe17Jf1A0kjtrndGxLOdahRop5UrVxbrM2fOLNa3bdtWrPfjPPtEvlTziaSfRMQrtr8i6WXbq2u1X0TEg51rD0C7TGR99p2Sdtauf2R7s6RTO90YgPY6qPfstqdJ+qakP9c23Wx7ve3HbZ9YZ8x821Xb1ZGRkfHuAqALJhx228dK+oOkH0fEh5J+JWm6pFkaPfL/bLxxETEYEZWIqDQ6ZxiAzplQ2G0frtGg/y4inpKkiHgvIj6NiM8k/VrSuZ1rE0CrGobdtiU9JmlzRPx8zPapY+72XUkb298egHaZyKfxF0iaJ2mD7eHatjslzbU9S1JI2i7phx3pEOiA4447rlh/8803u9RJ90zk0/g/SfI4JebUgUMI36ADkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4k4Yjo3s7sEUlvjdk0WdLurjVwcPq1t37tS6K3ZrWztzMiYtzzv3U17F/YuV2NiErPGijo1976tS+J3prVrd54GQ8kQdiBJHod9sEe77+kX3vr174kemtWV3rr6Xt2AN3T6yM7gC4h7EASPQm77Uts/4/tbbYX9KKHemxvt73B9rDtao97edz2Ltsbx2ybZHu17a21y3HX2OtRb/fa/mvtuRu2fVmPejvN9gu2N9veZPtHte09fe4KfXXleev6e3bbh0l6TdJsSTskrZM0NyJe7WojddjeLqkSET3/AobtCyX9TdJvI+IbtW3/LmlPRCyq/Ud5YkT8a5/0dq+kv/V6Ge/aakVTxy4zLukKSf+iHj53hb6uVheet14c2c+VtC0i3oiIv0taIWlOD/roexHxkqQ9B2yeI2lJ7foSjf5j6bo6vfWFiNgZEa/Urn8kaf8y4z197gp9dUUvwn6qpL+Mub1D/bXee0j6o+2Xbc/vdTPjmBIRO6XRfzySTu5xPwdquIx3Nx2wzHjfPHfNLH/eql6EfbylpPpp/u+CiPiWpEsl3VR7uYqJmdAy3t0yzjLjfaHZ5c9b1Yuw75B02pjbX5X0Tg/6GFdEvFO73CVplfpvKer39q+gW7vc1eN+/l8/LeM93jLj6oPnrpfLn/ci7OsknWn7a7aPkPR9SUM96OMLbB9T++BEto+R9B3131LUQ5Kur12/XtLTPezlc/plGe96y4yrx89dz5c/j4iu/0m6TKOfyL8u6d960UOdvr4u6b9rf5t63Zuk5Rp9Wfe/Gn1FdKOkkyStlbS1djmpj3r7T0kbJK3XaLCm9qi3f9LoW8P1koZrf5f1+rkr9NWV542vywJJ8A06IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUji/wATqjr/Bif/2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "digit  = x_test[7]\n",
    "plt.imshow(digit,cmap = plt.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Actual Label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test[7]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#x_train[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Normalizing Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Changing Dimension from 3 into 2\n",
    "\n",
    "As you can see above there are multiple values between 0 to 255 and 255 is Max value so what we're doing is managing the values between 0 and 1 by dividing the train sets by 255."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_x_train = x_train.reshape(60000, 28 * 28)\n",
    "new_x_test = x_test.reshape(10000, 28 * 28)\n",
    "new_x_train, new_x_test = new_x_train/255.0, new_x_test/255.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#x_train[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# One hot encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.utils import to_categorical\n",
    "\n",
    "y_train = to_categorical(y_train) \n",
    "y_test = to_categorical(y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n",
      "2\n",
      "Train features: (60000, 784)\n",
      "Train Labels:   (60000, 10)\n",
      "Test Features:  (10000, 784)\n",
      "Test Labels:    (10000, 10)\n"
     ]
    }
   ],
   "source": [
    "print(new_x_train.ndim)\n",
    "print(new_x_test.ndim)\n",
    "print(\"Train features:\",new_x_train.shape)\n",
    "print(\"Train Labels:  \",y_train.shape)\n",
    "print(\"Test Features: \",new_x_test.shape)\n",
    "print(\"Test Labels:   \",y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 48000 samples, validate on 12000 samples\n",
      "Epoch 1/5\n",
      "48000/48000 [==============================] - 9s 187us/sample - loss: 0.2516 - accuracy: 0.9263 - val_loss: 0.1282 - val_accuracy: 0.9621\n",
      "Epoch 2/5\n",
      "48000/48000 [==============================] - 8s 161us/sample - loss: 0.0967 - accuracy: 0.9710 - val_loss: 0.0930 - val_accuracy: 0.9701\n",
      "Epoch 3/5\n",
      "48000/48000 [==============================] - 9s 178us/sample - loss: 0.0648 - accuracy: 0.9802 - val_loss: 0.0960 - val_accuracy: 0.9712\n",
      "Epoch 4/5\n",
      "48000/48000 [==============================] - 7s 153us/sample - loss: 0.0489 - accuracy: 0.9845 - val_loss: 0.0969 - val_accuracy: 0.9710\n",
      "Epoch 5/5\n",
      "48000/48000 [==============================] - 7s 149us/sample - loss: 0.0374 - accuracy: 0.9879 - val_loss: 0.0880 - val_accuracy: 0.9761\n"
     ]
    }
   ],
   "source": [
    "models = tf.keras.Sequential\n",
    "layers = tf.keras.layers\n",
    "\n",
    "model = models([layers.Dense(512,activation=\"relu\",input_shape=(28 * 28,)),\n",
    "                layers.Dense(64,activation=\"relu\"),\n",
    "                layers.Dense(16,activation=\"relu\"),\n",
    "                layers.Dense(10, activation='softmax')])\n",
    "\n",
    "model.compile(optimizer=\"adam\", # adam # rmsprop\n",
    "             loss=\"categorical_crossentropy\",\n",
    "             metrics=[\"accuracy\"])\n",
    "\n",
    "history = model.fit(new_x_train,y_train,epochs=5,validation_split=(0.2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/1 - 1s - loss: 0.0394 - accuracy: 0.9781\n",
      "\n",
      "Loss:  7.851570602597785 %\n",
      "Accuracy:  97.81000018119812 %\n"
     ]
    }
   ],
   "source": [
    "evaluation = model.evaluate(new_x_test,y_test,verbose=2)\n",
    "print()\n",
    "print(\"Loss: \",evaluation[0]*100,\"%\")\n",
    "print(\"Accuracy: \",evaluation[1]*100,\"%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted value:  9\n",
      "Actual value:  9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAOFElEQVR4nO3dbazU5ZnH8d9vXY1P9QE5IrEqLdFE0ghtJsboxripNGpMsMZqiUE3mtIXPrSmJouuWqMvwI1tU4Opnq5auhRIpRJPoq4FYjR90zCaszwIK6hoqSgHMWhfYFe99sUZNkc8c89hnvH6fpKTmflfc8//yoQf/5m5Z/63I0IAvvz+odcNAOgOwg4kQdiBJAg7kARhB5L4x27ubPLkyTFt2rRu7hJIZfv27dq9e7fHq7UUdtuXSPqlpMMk/UdELCrdf9q0aapWq63sEkBBpVKpW2v6ZbztwyQ9LOlSSTMkzbU9o9nHA9BZrbxnP1fStoh4IyL+LmmFpDntaQtAu7US9lMl/WXM7R21bZ9je77tqu3qyMhIC7sD0IpWwj7ehwBf+O5tRAxGRCUiKgMDAy3sDkArWgn7Dkmnjbn9VUnvtNYOgE5pJezrJJ1p+2u2j5D0fUlD7WkLQLs1PfUWEZ/YvlnS8xqdens8Ija1rTMAbdXSPHtEPCvp2Tb1AqCD+LoskARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IImWlmy2vV3SR5I+lfRJRFTa0RSA9msp7DX/HBG72/A4ADqIl/FAEq2GPST90fbLtuePdwfb821XbVdHRkZa3B2AZrUa9gsi4luSLpV0k+0LD7xDRAxGRCUiKgMDAy3uDkCzWgp7RLxTu9wlaZWkc9vRFID2azrsto+x/ZX91yV9R9LGdjUGoL1a+TR+iqRVtvc/zrKI+K+2dIWD8uGHH9atLViwoDh206ZNxfqaNWuK9cMPP7xYR/9oOuwR8YakmW3sBUAHMfUGJEHYgSQIO5AEYQeSIOxAEu34IQw6bOnSpcX6XXfdVbf29ttvt7Tv0rSeJJ100kktPT66hyM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBPHsf2LFjR7F+2223Feu7d9c/32ftJ8hNu+WWW4r1xYsXF+uTJk1qaf9oH47sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE8+x94MEHHyzW33///S518kUrVqwo1p977rlivfRb+0Zz+EcccUSxjoPDkR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCevQveeuutYv2JJ55o6fFnzqy/mO6UKVOKY1evXt3Svvfu3Vusl75DcO211xbHnnLKKU31hPE1PLLbftz2Ltsbx2ybZHu17a21yxM72yaAVk3kZfxvJF1ywLYFktZGxJmS1tZuA+hjDcMeES9J2nPA5jmSltSuL5F0RZv7AtBmzX5ANyUidkpS7fLkene0Pd921XZ1ZGSkyd0BaFXHP42PiMGIqEREZWBgoNO7A1BHs2F/z/ZUSapd7mpfSwA6odmwD0m6vnb9eklPt6cdAJ3ScJ7d9nJJF0mabHuHpJ9KWiTp97ZvlPS2pO91sslD3fDwcLHeaA30Cy+8sFh/8cUX69b27dtXHLts2bJifeHChcX6tm3bivV33323bm3OnDnFsY1+K8856Q9Ow7BHxNw6pW+3uRcAHcTXZYEkCDuQBGEHkiDsQBKEHUiCn7h2wccff1ysN1pWudGSzSVHHnlksX7DDTcU6ytXrizWX3/99WI9IurWjj766OJYTiXdXhzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJ5tm7YPny5S2Nf+aZZ4r1K67o3CkAq9Vqxx77vPPOK9aPPfbYju07I47sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE8+xdMHduvRP0jnr66fJp99etW1esb9mypW5tw4YNxbGrVq0q1j/44INi/YQTTmh6/ODgYHHsvHnzivUZM2YU6/g8juxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATz7F1w8cUXF+vHH398sb5+/fpi/eyzz65ba3RO+kZmz55drD/88MPF+uWXX1639tprrxXHPvTQQ8X6I488Uqzj8xoe2W0/bnuX7Y1jtt1r+6+2h2t/l3W2TQCtmsjL+N9IumSc7b+IiFm1v2fb2xaAdmsY9oh4SdKeLvQCoINa+YDuZtvray/zT6x3J9vzbVdtV0dGRlrYHYBWNBv2X0maLmmWpJ2SflbvjhExGBGViKgMDAw0uTsArWoq7BHxXkR8GhGfSfq1pHPb2xaAdmsq7Lanjrn5XUkb690XQH9oOM9ue7mkiyRNtr1D0k8lXWR7lqSQtF3SDzvY4yFv0qRJxfqTTz5ZrF911VXF+t69e+vWSuujS9Ktt95arD/wwAPFeqP136+88sq6tYULFxbHPv/888V6o7Xhp0+fXqxn0zDsETHemRce60AvADqIr8sCSRB2IAnCDiRB2IEkCDuQBD9x7QONfgK7cuXKYn3ZsmV1a41O9XzfffcV642m1hq5++6769Y2b95cHNvoFNuNel+yZEmxng1HdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1Ignn2Q0CjefhG9V466qij6tauueaa4thG8+wvvPBCsb5nT/1TJzb62fGXEUd2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCeXb0zNVXX12sDw0NFesrVqwo1hcvXly3ds899xTHfhlxZAeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJNxoSd92qlQqUa1Wu7Y/HNqGh4eL9fPPP79Y37dvX93ali1bimPPOuusYr1fVSoVVatVj1dreGS3fZrtF2xvtr3J9o9q2yfZXm17a+3yxHY3DqB9JvIy/hNJP4mIsyWdJ+km2zMkLZC0NiLOlLS2dhtAn2oY9ojYGRGv1K5/JGmzpFMlzZG0f32dJZKu6FSTAFp3UB/Q2Z4m6ZuS/ixpSkTslEb/Q5B0cp0x821XbVdHRkZa6xZA0yYcdtvHSvqDpB9HxIcTHRcRgxFRiYjKwMBAMz0CaIMJhd324RoN+u8i4qna5vdsT63Vp0ra1ZkWAbRDw5+42rakxyRtjoifjykNSbpe0qLaZfm8v8BBmjVrVrF+//33F+u333573dodd9xRHLt06dJivXSK7H41kd+zXyBpnqQNtvdPfN6p0ZD/3vaNkt6W9L3OtAigHRqGPSL+JGncSXpJ325vOwA6ha/LAkkQdiAJwg4kQdiBJAg7kASnksYh67rrrivWH3300bq1p556qm5NkrZu3Vqsn3POOcV6P+LIDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM+OQ1ajMx+tWbOmbu2MM84ojl20aFGxvmzZsmK9H3FkB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkmGfHl9bpp59etzZ79uzi2KGhoWL91VdfLdZnzJhRrPcCR3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSGIi67OfJum3kk6R9JmkwYj4pe17Jf1A0kjtrndGxLOdahRop5UrVxbrM2fOLNa3bdtWrPfjPPtEvlTziaSfRMQrtr8i6WXbq2u1X0TEg51rD0C7TGR99p2Sdtauf2R7s6RTO90YgPY6qPfstqdJ+qakP9c23Wx7ve3HbZ9YZ8x821Xb1ZGRkfHuAqALJhx228dK+oOkH0fEh5J+JWm6pFkaPfL/bLxxETEYEZWIqDQ6ZxiAzplQ2G0frtGg/y4inpKkiHgvIj6NiM8k/VrSuZ1rE0CrGobdtiU9JmlzRPx8zPapY+72XUkb298egHaZyKfxF0iaJ2mD7eHatjslzbU9S1JI2i7phx3pEOiA4447rlh/8803u9RJ90zk0/g/SfI4JebUgUMI36ADkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4k4Yjo3s7sEUlvjdk0WdLurjVwcPq1t37tS6K3ZrWztzMiYtzzv3U17F/YuV2NiErPGijo1976tS+J3prVrd54GQ8kQdiBJHod9sEe77+kX3vr174kemtWV3rr6Xt2AN3T6yM7gC4h7EASPQm77Uts/4/tbbYX9KKHemxvt73B9rDtao97edz2Ltsbx2ybZHu17a21y3HX2OtRb/fa/mvtuRu2fVmPejvN9gu2N9veZPtHte09fe4KfXXleev6e3bbh0l6TdJsSTskrZM0NyJe7WojddjeLqkSET3/AobtCyX9TdJvI+IbtW3/LmlPRCyq/Ud5YkT8a5/0dq+kv/V6Ge/aakVTxy4zLukKSf+iHj53hb6uVheet14c2c+VtC0i3oiIv0taIWlOD/roexHxkqQ9B2yeI2lJ7foSjf5j6bo6vfWFiNgZEa/Urn8kaf8y4z197gp9dUUvwn6qpL+Mub1D/bXee0j6o+2Xbc/vdTPjmBIRO6XRfzySTu5xPwdquIx3Nx2wzHjfPHfNLH/eql6EfbylpPpp/u+CiPiWpEsl3VR7uYqJmdAy3t0yzjLjfaHZ5c9b1Yuw75B02pjbX5X0Tg/6GFdEvFO73CVplfpvKer39q+gW7vc1eN+/l8/LeM93jLj6oPnrpfLn/ci7OsknWn7a7aPkPR9SUM96OMLbB9T++BEto+R9B3131LUQ5Kur12/XtLTPezlc/plGe96y4yrx89dz5c/j4iu/0m6TKOfyL8u6d960UOdvr4u6b9rf5t63Zuk5Rp9Wfe/Gn1FdKOkkyStlbS1djmpj3r7T0kbJK3XaLCm9qi3f9LoW8P1koZrf5f1+rkr9NWV542vywJJ8A06IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUji/wATqjr/Bif/2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "predict = 7\n",
    "a = model.predict_classes(new_x_test)[predict]\n",
    "print(\"Predicted value: \",a)\n",
    "print(\"Actual value: \",np.argmax(y_test[predict]))\n",
    "pic  = x_test[predict]\n",
    "plt.imshow(pic,cmap = plt.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convnets for computer vision tasks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following lines of code show what a basic convnet looks like. It’s a stack of\n",
    "Conv2D and MaxPooling2D layers.\n",
    "The number of channels is controlled by the first\n",
    "argument passed to the Conv2D layers (32 or 64)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.datasets import mnist\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train features: (60000, 28, 28, 1)\n",
      "Test Features:  (10000, 28, 28, 1)\n"
     ]
    }
   ],
   "source": [
    "# normalizing data\n",
    "x_train = x_train.reshape((60000, 28, 28, 1))\n",
    "x_test = x_test.reshape((10000, 28, 28, 1))\n",
    "new_x_train, new_x_test = x_train/255.0, x_test/255.0\n",
    "#new_x_train, new_x_test = x_train.astype('float32') / 255, x_test.astype('float32') / 255\n",
    "\n",
    "print(\"Train features:\",new_x_train.shape)\n",
    "print(\"Test Features: \",new_x_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train Labels:   (60000, 10)\n",
      "Test Labels:    (10000, 10)\n"
     ]
    }
   ],
   "source": [
    "# one hot encol=ding\n",
    "from tensorflow.keras.utils import to_categorical\n",
    "y_train = to_categorical(y_train) \n",
    "y_test = to_categorical(y_test)\n",
    "\n",
    "print(\"Train Labels:  \",y_train.shape)\n",
    "print(\"Test Labels:   \",y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d (Conv2D)              (None, 26, 26, 32)        320       \n",
      "_________________________________________________________________\n",
      "max_pooling2d (MaxPooling2D) (None, 13, 13, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_1 (Conv2D)            (None, 11, 11, 64)        18496     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 5, 5, 64)          0         \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 3, 3, 64)          36928     \n",
      "=================================================================\n",
      "Total params: 55,744\n",
      "Trainable params: 55,744\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "models = tf.keras.Sequential\n",
    "layers = tf.keras.layers\n",
    "\n",
    "model1 = models([layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),\n",
    "                layers.MaxPooling2D((2, 2)),\n",
    "                layers.Conv2D(64, (3, 3), activation='relu'),\n",
    "                layers.MaxPooling2D((2, 2)),\n",
    "                layers.Conv2D(64, (3, 3), activation='relu')])\n",
    "\n",
    "model1.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to feed the last output tensor (of shape (3, 3, 64)) into a densely\n",
    "connected classifier with a stack of Dense layers. These classifiers process vectors, which are 1D, whereas the current output is a 3D tensor. First we have to flatten the 3D outputs to 1D, and then add a few Dense layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_3 (Conv2D)            (None, 26, 26, 32)        320       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 13, 13, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_4 (Conv2D)            (None, 11, 11, 64)        18496     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_3 (MaxPooling2 (None, 5, 5, 64)          0         \n",
      "_________________________________________________________________\n",
      "conv2d_5 (Conv2D)            (None, 3, 3, 64)          36928     \n",
      "_________________________________________________________________\n",
      "flatten (Flatten)            (None, 576)               0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 64)                36928     \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 93,322\n",
      "Trainable params: 93,322\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "models = tf.keras.Sequential\n",
    "layers = tf.keras.layers\n",
    "\n",
    "model2 = models([layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),\n",
    "                layers.MaxPooling2D((2, 2)),\n",
    "                layers.Conv2D(64, (3, 3), activation='relu'),\n",
    "                layers.MaxPooling2D((2, 2)),\n",
    "                layers.Conv2D(64, (3, 3), activation='relu'),\n",
    "                layers.Flatten(),\n",
    "                layers.Dense(64, activation='relu'),\n",
    "                layers.Dense(10, activation='softmax')])\n",
    "\n",
    "model2.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "model2.compile(optimizer='rmsprop',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples\n",
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 57s 947us/sample - loss: 0.1752 - accuracy: 0.9453\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 56s 926us/sample - loss: 0.0469 - accuracy: 0.9852\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 54s 904us/sample - loss: 0.0322 - accuracy: 0.9900\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 54s 897us/sample - loss: 0.0235 - accuracy: 0.9927\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 54s 902us/sample - loss: 0.0197 - accuracy: 0.9938\n"
     ]
    }
   ],
   "source": [
    "history = model2.fit(new_x_train, y_train, epochs=5, batch_size=64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/1 - 5s - loss: 0.0153 - accuracy: 0.9910\n",
      "\n",
      "Loss:  3.0068729236282525 %\n",
      "Accuracy:  99.09999966621399 %\n"
     ]
    }
   ],
   "source": [
    "# model of chapter 5\n",
    "evaluation1 = model2.evaluate(new_x_test,y_test,verbose=2)\n",
    "print()\n",
    "print(\"Loss: \",evaluation1[0]*100,\"%\")\n",
    "print(\"Accuracy: \",evaluation1[1]*100,\"%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Whereas the densely connected network from chapter 2 had a test accuracy of 97.8%,\n",
    "the basic convnet has a test accuracy of 99.0% and we decreased the error rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss:  7.851570602597785 %\n",
      "Accuracy:  97.81000018119812 %\n"
     ]
    }
   ],
   "source": [
    "# First model of chapter 2\n",
    "print(\"Loss: \",evaluation[0]*100,\"%\")\n",
    "print(\"Accuracy: \",evaluation[1]*100,\"%\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
